MOBILE ROBOT, AND SYSTEM AND METHOD FOR AUTONOMOUS 

NAVIGATION OF THE SAME 



BACKGROUND OF THE INVENTION 

5 

This application claims the priority of Korean Patent Application No. 10-2003- 
0001828 filed on January 11, 2003 in the Korean Intellectual Property Office, the 
disclosure of which is incorporated herein in its entirety by reference. 



10 1 . Field of the Invention 

The present invention relates generally to a mobile robot, and system and 
method for autonomous navigation of the same, and more particularly to a mobile 
robot, and system and method for autonomous navigation of the same, in which the 
mobile robot controls the flickering of the light sources of a landmark array disposed 
15 in a working space and ascertains its position using one or more flicking light sources, 
thus enabling the autonomous navigation of the mobile robot. 
2. Description of the Related Art 

In general, an autonomous mobile robot should precisely recognize its current 
pose and move firom the current position to a target position (destination) in order to 
20 freely move across a certain area. 

To this end, a certain landmark array is employed as a means for providing 
world coordinates to allow the mobile robot to recognize its current position. The 
mobile robot recognizes the landmark array disposed in a certain working space 
through the signal processing of an image captured by a camera and ascertains its 
25 current position using the recognized landmark array. 

Such a landmark array may be designed in various patterns. Generally, a 
certain geometrical pattern, such as a circle and a chain, is employed as the form of 
the landmark array. 
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Accordingly, the mobile robot extracts the certain pattern of a landmark array 
from an image signal for a working space. In this case, the process of extracting the 
pattern of the landmark array from the image signal is greatly affected by extemal 
environmental factors, such as the illumination of a working space captured by a 
5 camera. 

That is, the recognition of the pattern of a landmark array from an image 
captured by a camera is greatly affected by extemal factors such as illumination, the 
shapes of molded objects, the patterns of wallpaper, etc. Accordingly, the operation 
process of recognizing the pattern of the landmark array becomes further complicated. 

10 Additionally, since relatively long processing time is required to recognize the 

position of the mobile robot using the landmark array due to the complicated 
operation process, it is difficult for autonomous navigation of the mobile robot to be 
performed in real-time. 

Further, in a home network environment that has been developed and widely 

15 implemented, in which all home appliances are controlled by using a mobile robot, it 
is not desirable in terms of appearance to install a certain pattern of a landmark array 
in the interior of a house. 

Accordingly, there have been demands for a mobile robot and system for 
autonomous navigation of the mobile robot that is capable of autonomous moving 

20 through the use of a landmark array applicable to a home network environment and 
suitable for application to actual life due to adaptability to the variations of an extemal 
environment, such as illumination. 

SUMMARY OF THE INVENTION 

25 

Accordingly, an object of the present invention is to provide a mobile robot, 
and system and method for autonomous navigation of the same, in which the mobile 
robot can ascertain its position by controlling the flickering of the light sources of a 
landmark array disposed in a working space, and can autonomously move to a 
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destination based on the ascertained position. 

Another object of the present invention is to provide a mobile robot, and 
system and method for autonomous navigation of the same, in which the mobile robot 
is capable of efficiently calculating its pose(position and heading angle) by selecting a 

5 light source suitable for the calculation of the position with reference to previously 
stored world coordinates of the light sources of a landmark array. 

Yet another object of the present invention is to provide a means for providing 
world coordinates for autonomous navigation of the mobile robot, which uses a 
plurality of light sources having a certain wavelength as a landmark array, thus being 

10 suitable for a home network environment. 

In order to accomplish the above object, the present invention provides a 
mobile robot, including a communication module for transmitting a light source 
control signal to selectively control the flickering of the light sources of a landmark 
array provided in a working space, an image processing module for calculating image 

15 coordinates of the flickering of the light sources by detecting the flickering of the light 
sources, controlled by a the light source control signal, a pose calculation module for 
calculating position coordinates of the mobile robot using the calculated image 
coordinates and previously stored world coordinates of the light sources, a move 
control module for calculating a moving path for the mobile robot by applying the 

20 position coordinates of the mobile robot to previously stored spatial coordinates of the 
working space and controlling the mobile robot to move along the moving path, and a 
main control module for controlling interoperations of the modules and general 
operations of the mobile robot. 

Preferably, the mobile robot may further comprise a memory module for 

25 storing the world coordinates of the light sources, spatial coordinates of the mobile 
robot in the working space, and parameters calculated through camera calibration for 
compensating for distortion of the camera lens. 

The camera may be equipped with a filter for filtering a certain wavelength of 
the light source. 
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In addition, the present invention provides a system for autonomous navigation 
of a mobile robot, including a landmark array comprising a plurality of light sources 
disposed in a certain area to selectively flicker, a landmark array control module for 
controlling the flickering of the light sources of the landmark array, and a mobile 

5 robot equipped with a pose calculation module for selectively controlling the 
flickering of the light sources of the landmark array by tremsmitting a light source 
control signal to the landmark array control module and recognizing a pose of the 
mobile robot using the flickering light sources. 

In addition, the present invention provides a method for autonomous 

10 navigation of a mobile robot, including the first step of the mobile robot selectively 
controlling the flickering of the flickering of the light sources of a landmark array 
arranged across a certain working space, the second step of extracting image 
coordinates of the light sources by detecting a light source controlled to flicker from 
an image signal input from the camera, and the third step of calculating a current 

15 position of the mobile robot with reference to the image coordinates and previously 
stored world coordinates of the light sources. 

BRIEF DESCRIPTION OF THE DRAWINGS 

20 The above and other objects, features and advantages of the present invention 

will be more clearly understood from the following detailed description taken in 
conjunction with the accompanying drawings; in which: 

FIG. 1 is a schematic view showing the construction of a system for 
autonomous navigation of a mobile robot according to an embodiment of the present 

25 invention; 

FIG. 2 is a flowchart showing a method for autonomous navigation of a mobile 
robot according to an embodiment of the present invention; 

FIG. 3 is a perspective view of a landmark array disposed in a working space 
of a mobile robot according to an embodiment of the present invention; 
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FIG. 4 is a perspective view of the mobile robot having been moved to a 
certain position according to an embodiment of the present invention; 

FIG. 5 is a flowchart showing the process of determining an initial position 
according to an embodiment of the present invention; and 
5 FIG. 6 is a flowchart showing the process of calculating the position of the 

mobile robot according to an embodiment of the present invention. 

DESCRIPTION OF THE PREFERRED EMBODIMENTS 

10 Reference now should be made to the drawings, in which the same reference 

numerals are used throughout the different drawings to designate the same or similar 
components. 

The construction and operation of a mobile robot, and system and method for 
autonomous navigation of the same according to the present invention are described in 
15 detail with reference to the accompanying drawings. 

As shown in FIG. 1 , the system for autonomous navigation of the mobile robot 
includes a landmark array 100, a landmark array control module 300, and a mobile 
robot 500. 

The landmark array 100 is made up of a plurality of light sources that are 
20 arranged across a certain area and can flicker. The light sources are formed of light 
emitting devices, including electro-luminescent devices, such as organic electro- 
luminescent devices, inorganic electro-luminescent devices, or Light Emitting Diodes 
(LEDs), which emit light with a certain wavelength and a certain brightness. 

Meanwhile, the light sources are each assigned with position information 
25 comprised of a specific identification number and world coordinates in a working 
space in which the landmark array is arranged. 

The landmark array control module 300 controls corresponding light sources 
of the landmark array 100 to flicker in response to a flickering control signal 
transmitted from the mobile robot 500. The landmark array control module 300 
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includes an access point (AP) 301 that is one of component devices constituting a 
wireless local area network and receives and processes the light source control signal 
transmitted from the mobile robot 500, and a light source control unit 302 that 
controls corresponding light sources to flicker in response to the light source control 

5 signal input from the access point 301 . 

The mobile robot 500 is constructed to include a pose calculation module 504 
that selectively controls the flickering of the light sources of the landmark array 100 
by transmitting a light source control signal to the landmark array control module 300 
and recognizes the position thereof using the flickering light sources, thus enabling 

10 autonomous navigation. The mobile robot 500 further includes a main control module 
501, a communications module 502, an image processing module 503, a motion 
control module 505, and a memory module 506. 

The main control module 501 controls the entire operations of position 
calculation and moving according to an operation management algorithm for 

15 autonomous navigation of the mobile robot 500, £ind functions to control the 
interoperations of respective modules. 

The communications module 502 transmits a light source control signal to 
control the light sources of the landmark array 100 to flicker under the operation 
control of the main control module 501. And the previously stored world coordinates 

20 of the light source are refrenced by the communication module 502. Wireless 
communications performed to transmit/receive the light source control signal between 
the access point 301 and the communications module 502 is implemented in such a 
way that the access point 301 and the communications module 502 are newly assigned 
with frequencies of a high bandwidth, and transmit/receive data via the assigned 

25 frequencies. 

Additionally, such wireless communications may be implemented by wireless 
communications protocols, such as Infrared Data Association (IrDA) using infrared 
radiation or Bluetooth that is a wireless local area communications technology. 

The image processing module 503 processes a signal to detect feature points of 



the specifiedlight source, controlled to flicker via the communications module 502, 
from an image signal obtained by a camera. The image processing module 503 
includes a camera (not shown) equipped with a filter for filtering the image signal 
based on the wavelength of light sources, and a signal processing unit (not shown) for 

5 detecting feature points corresponding to the wavelength of the light sources from the 
image signal filtered by the camera, and for extracting the coordinates of the detected 
feature points (hereinafter referred to as "image coordinates"). 

The method of detecting feature points corresponding to the wavelength of 
light sources in the signal process xmit and extracting image coordinates can be 

10 implemented through thresholding technology and grouping & labeling technology (4- 
connectivity) (Digital Image Processing; Rafael C. Gonzales, Richard E. Wodds, 
Addison Wesley). This method is performed by binarizing the image signal filtered 
using the wavelength of the light sources, extracting a portion, on which pixels having 
a value equal to or greater than a certain value (for example, 50) are connected to each 

15 other and which has a gray level of 255, as the region of a light source, detecting the 
light source by removing noise components from the extracted region and extracting 
image coordinates based on the center of gravity. 

The pose calculation module 504 calculates the precise coordinates of the 
mobile robot 500 with reference to image coordinates for the specified light source 

20 obtained through the image processing module 503 and the world coordinates of the 
certain light source detected through the memory module 506. The translation and 
rotation of the mobile robot 500 are calculated by applying the image coordinates and 
the world coordinates to the specified position calculation algorithm. 

The position calculation algorithm constructs an extension model for obtaining 

25 the translation and rotation of the camera using a world coordinate system and a 
camera coordinate system, and obtaining a certain transformation formula for 
calculating the translation and rotation of the camera by applying the extension model 
to a formula for compensating for distortion caused by the lens of the camera. 

The pose calculation module 504 determines the current pose of the mobile 
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robot 500 by calculating a translation that is a distance by which the camera (that is, 
the mobile robot 500) is spaced apart from the specified light source and a rotation 
with respect to the light source by applying the image coordinates and world 
coordinates of the light source to the transformation formula obtained through the 
5 pose calculation algorithm. 

The motion control module 505 controls the motion of the mobile robot 500 
using a drive me£ins for driving right and left wheel motors to allow the mobile robot 
500 to autonomously move. The motion control module 505 determines a moving 
path, along which the robot 500 moves to a destination using current position 

10 coordinates, by matching the translation and rotation of the mobile robot 500 output 
from the pose calculation module 504 with the area coordinate information of a 
working space previously stored in the memory module 506, and controls the mobile 
robot 500 to move along the determined moving path. 

The memory module 506 stores an operation management algorithm for 

15 autonomous navigation of the mobile robot 500 and information for the operation 
management algorithm. The memory module 506 stores parameters for compensating 
for the distortion of the lens of the camera calculated through camera calibration, the 
world coordinates of the light sources of the landmark array 100, and the coordinate 
information of the working space across which the mobile robot 500 autonomously 

20 navigates. 

1 The method for autonomous navigation of the mobile robot using the system 

for autonomous navigation of the mobile robot according to the present invention is 
described in detail with reference to the accompanying drawings. 

As shown in FIG. 2, the main control module 501 of the mobile robot 500 
25 transmits a light source control signal (a tum on signal) to turn on the specified light 
sources of the landmark array 100 at step SI. 

As the light source control signal transmitted from the mobile robot 500 
through the access point 301 of the landmark array control module 300 is received 
through the access point 301 of the landmark array control module 300, a 



corresponding light source is turned on by the light source control vinit 302, and the 
image processing module 503 of the mobile robot 500 determines whether the light 
source controlled to flicker in response to the light source control signal is detected by 
detecting the feature points of the light source through an image signal captured by the 

5 camera at step S2. 

If the light source is not detected as the result of the determination of step S2, a 
light source is detected by selecting a neighboring light source with reference to the 
world coordinates of the light sources of the landmark array 100 previously stored in 
the memory module 506, transmitting a light source control signal to turn on this light 

10 source, and determining whether the light source is detected through the camera at 
step S3. 

If the light source is detected as the result of the determination of step 83, the 
current position of the mobile robot 500 is determined using the image coordinates of 
the light source detected by the camera and the world coordinates of the light source 
15 detected by the memory module 506 at step S4. 

The main control module 501 of the mobile robot 500 determines a moving 
path of the mobile robot 500 using current world coordinates determined through the 
above-described step, and controls the mobile robot 500 to move along the moving 
path determined by the motion control module 505. 
20 The method for autonomous navigation of a mobile robot according to the 

present invention is described in detail with reference to the accompanying drawing. 

If a house equipped with a home automation system is selected as the working 
space of the mobile robot, the light sources of the landmark array 100 are installed 
throughout the house as shown in FIG. 3 and controlled to flicker through the 
25 landmark array control module 300 provided in the server of the home automation 
system. 

' When the light source is tumed on in the state where the mobile robot 500 is 

moved to a random position in the working space of the mobile robot 500, as shown in 
FIG. 4, the mobile robot 500 roughly determines the current position thereof through 
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an initializing process and precisely calculates the current position thereof through a 
localization process. 

The initialization process is the process of determining the initial position of 
the mobile robot as power is applied to the mobile robot, which is performed as shown 
5 in FIG. 5. 

When power is applied to the mobile robot 500 at step Sll, the main control 
module 501 of the mobile robot 500 sequentially controls the light sources of the 
landmark array 100 to flicker with reference to the specified light source and the 
world coordinates of the light sources of the landmark array 100 at step SI 2. 

10 In the process of controlling the light sources of the landmark array 100 to 

sequentially flicker, the main control module of the mobile robot 500 detects the 
feature points of the light source from an image signal obtained by the camera using 
the image processing module 503, and determines whether the light source controlled 
to flicker is detected by the camera at step SI 3. 

15 If the light source is not detected as the result of the determination of step SI 3, 

the process of controlling the light sources of the landmark array 100 to sequentially 
flicker after changing the direction of the camera by rotating the camera, and detecting 
the light source using an image obtained by the camera is repeated a set number of 
times Max_Num at step SI 4. If the light source is not detected after the process is 

20 repeated a randomly set number of times, the process is determined to be an error by 
generating a light source detection error signal at step SI 5. 

If the light source is detected as the result of the determination of step SI 3, the 
world coordinates of the detected light source stored in the memory module 506 is 
detected through the use of the ID number of the detected light source at step SI 6, and 

25 the current position of the mobile robot 500 is roughly ascertained through the use of 
the world coordinates of the detected light source at step SI 7. 

For example, in the case where the world coordinates of the light source 
ascertained through the use of the ID number of the light source detected by the 
camera during the flickering of the light source of the landmark array 100 is the 



position information of a living room, the main control module 501 of the mobile 
robot 500 can roughly ascertain that the mobile robot 500 is positioned in the living 
room. 

When the initialization process of the mobile robot 500 is completed, the main 
5 control module 501 of the mobile robot 500 performs a position calculation process of 
calculating the precise position of the mobile robot 500 with reference to the rough 
position information obtained through the initialization process. 

The position calculation process is performed as shown in FIG. 6. First, the 
main control module 501 of the mobile robot 500 sets the number of light sources, 
10 which is required to calculate the precise position of the mobile robot 500, to a certain 
number N. 

After the number of light sources N is set to calculate the precise position, N 
light sources, which are suitable for the position calculation, are selected from the 
light sources near the light source detected during the initialization process at step 
15 S21, and a light source control signal is transmitted to tum on the selected N light 
sources at step S22. 

For example, when the light source detected during the initialization process is 
a fifth light source of the living room in the case where the number of light sources 
required to calculate the position of the mobile robot 500 is set to five, light sources of 

20 a set number set to calculate the position can be turned on by turning on both two light 
sources on the right side of the fifth light source and two light sources on the left side 
of the fifth light source, or by turning on four light sources on the right side of the fifth 
light source or four light sources on the left side of the fifth light source. 

The landmark array control module 300 turns on a corresponding light source 

25 in response to a transmitted light source control signal, and the main control module 
501 of the mobile robot 500 detects the feature points of the light source from em 
image signal obtained by the camera using the image processing module 503, and 
determines whether the N light sources turned on to detect the position of the robot 
500 are detected at step S23. 
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If the N light sources are not detected as the result of the determination of step 
S24, it is determined whether the number of light sources detected by the camera is 
equal to or more than two at step S24. If the number of the detected light sources is 
less than two, a light source detected through the camera is detected by selecting and 
5 flickering another light source at step S22. If the number of the detected light sources 
is two or more at step S24, the ID numbers of the light sources detected through the 
camera are ascertained by controlling light sources to flicker at step S25. 

The image coordinates of N light sources and two or more light sources 
detected through the above-described determination are extracted from an image 
10 signal obtained by the camera, and world coordinates corresponding to light sources is 
detected from the memory module 506 using the ascertained IDs at step S26. 

The pose calculation module 504 calculates the translation and rotation of the 
mobile robot 500 by applying the image coordinates and world coordinates of the 
detected light sources to the transformation formula obtained through the position 
1 5 calculation algorithm. 

The main control module 501 of the mobile robot 500 determines a moving 
path for the mobile robot 500 by applying the translation and rotation of the mobile 
robot 500 to the spatial coordinate information of the mobile robot 500 in the working 
space, and controls the mobile robot 500 to move through the use of the motion 
20 control module 505. 

If the number of light sources required to calculate the position of the mobile 
robot 500 is set to two, the image coordinates of light sources detected through an 
image signal obtained by the camera in the image processing module 503 of the 

mobile robot 500 are (w,, v,) and (Wj^ ^2) > specific coordinates are 

25 {Xpf^^ , , Zjy^ ) and (Xjy^ , Y^^^ , Z^^^ ) , the translation and rotation of the mobile 

robot are obtained as described below. 

When a displacement matrix for obtaining the translation and rotation of the 
mobile robot is constructed using the world coordinate system and the camera 
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coordinate system, the camera of the mobile robot moves at a certain height and the 
translation with respect to a z axis is zero, the following displacement matrix is 
obtained. 



t = 
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5 where / is a translation and if is a rotation. 

When the displacement matrix with respect to the translation and rotation of 
the mobile robot is obtained, the displacement matrix is applied to a formula for 

compensating for distortion caused by the lens of the camera. The lens distortion 
compensation formula is as follows (see Three-Dimensional Computer Vision - A 
10 Geometric Viewpoint, Oliver Faugeras, 1993). 
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A displacement matrix with respect to translation and rotation is added to the 
lens distortion compensation formula. An extension model formed by including the 
translation and rotation matrix is as follows. 



m = A{R\t]P- 
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In the above matrix equation, fic, fy^ and are parameter values 

determined according to camera characteristics through camera calibration, and 
are image coordinates obtained from an input image signal with respect to the feature 
points of the detected light sources, and X^^ , 7^^^ and Z^^ are the world coordinates 

20 of detected light sources. 
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The above matrix equation is expressed by the equations for and v. as 

described below. 

^i^fx^w., cos^+/,y^ sin^ + WoZ«, 
v.^^f^Xy,^ sin(9 + /^y^ cos^ + VoZ^. +//^ 

The translation tx and ty and rotation 9 of the mobile robot are obtained with 
5 reference to the coordinate information (m, v,) and (^2, Vj) , and 

) and iXy^^ , 

^iv, 5 •^w, ) light sources detected by the camera as 

described below. 

"i - /r^>f, cos^ - /x^"^, sin 0 - UqZ^^ 



t = 



Vi + fyXyy sin 0 - f Y^ sin ^ - v^Z^ 



COS0 = 



fy 

"2 " fx i^w, - \ ) sin^ - u^jZy,^ - Zy,^ ) 



sin ^ = 



By the above process, the pose calculation module calculates the current 
10 position of the mobile robot 500 and eiscertains precise current position of the mobile 
robot 500 by applying the calculated translation and rotation values to spatial 
coordinates with respect to the working space of the mobile robot 500. 

If the position of the mobile robot is calculated through n light sources, 
equation 1 is converted into the following matrix equation. 
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The image coordinates and world coordinates of the n light sources are applied 
to the converted equation 2 as described below. 



cos^ 
sin^ 



AX = U:^X = {A'Ay'A'U 



5 With the above process, the current position of the mobile robot 500 is 

calculated. The main control module 501 of the mobile robot 500 ascertains the 
precise current position of the mobile robot 500 by applying the calculated translation 
and rotation values to the previously stored spatial coordinates with respect to the 
working space of the mobile robot 500. 

10 Additionally, a moving path for the mobile robot to move to a destination is 

determined using the position information, and the mobile robot 500 is controlled to 
move through the motion control module 505. 

In accordance with the present invention, the mobile robot can move to a 
precise destination by controlling the light sources of a landmark array to flicker and 

15 ascertaining its position. 

The mobile robot selects a certain light source useful to the calculation of its 
position from the light sources of the landmark array, and efficiently calculates its 
position with reference to the previously stored world coordinates of the light sources 
of the landmark array. 

20 Additionally, the landmark array of the present invention uses a plurality of 

light sources having a certain wavelength, so it can be used as a means for providing 
position information to aid the mobile robot in performing autonomous navigation in a 
house environment without being influenced by the variations of a house environment. 
Although the preferred embodiments of the present invention have been 

25 disclosed for illustrative purposes, those skilled in the art will appreciate that various 
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modifications, additions and substitutions are possible, without departing from the 
scope and spirit of the invention as disclosed in the accompanying claims. 
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